The present invention is directed to a method and system for converting two-dimensional (monoscopic) video to three-dimensional ‘(stereoscopic) video, and, more specifically, to a method and system for converting two-dimensional (monoscopic) video to three-dimensional (stereoscopic) video in real time.
Humans have a binocular vision system that uses two eyes spaced approximately two and a half inches (approximately 6.5 centimeters) apart. Each eye sees the world from a slightly different perspective. The brain uses the difference in these perspectives to calculate or gauge distance. This binocular vision system is responsible for the ability to determine with relatively good accuracy the distance of an object up to approximately twenty feet away. The relative distance of multiple objects in a field of view can also be determined. Using only one eye would significantly decrease the accuracy of this distance determination.
Traditional three-dimensional movies or videos (herein after referred to generically as “videos”) are made using two video sources (e.g. cameras) that are mounted side-by-side, about three (e.g. at the same distance as the separation of human eyes) to eight inches apart. This distance is often referred to as the interaxial or interoccular distance. The two video sources actually create two videos; one for the left eye and one for the right eye. Each video is made up of a series of “frames” (referred to as “frames” or “video frames”).
Traditionally, projection or display of a three-dimensional video has been accomplished by projecting or displaying a plurality of videos using, for example, a color differentiation system or a polarization system. This can be done using a plurality of projection or display devices (e.g. projectors) that each display one of the videos. This has also been done using single projection or display devices (e.g. the digital or analog display system of a computer, BETAMAX® player, VCR, DVD player, blue-ray player, television) that display the videos in an overlapping or interleaving fashion. The human binocular vision system is able to correlate these overlapping displays automatically because each eye sees only one of the overlapping or interleaving displays.
FIG. 1 shows an exemplary traditional color differentiation system 20 for projecting or displaying three-dimensional video 22 that uses color for differentiation. The first video source 24 projects through a first color filter (e.g. red) and a second video source 26 projects through a second color filter (e.g. blue). The video 22 in this figure is shown with an exaggerated double image. Viewers wear special glasses 28 with corresponding colored lenses 30, 32. For example, the first lens 30 would be a color that is the same color as one of the color filters (e.g. blue—shown as horizontal lines parallel to the bottom of the drawing page) and the second lens 32 would be a color that is the same color as the other color filter (e.g. red—shown as lines vertical lines parallel to the side of the drawing page). The screen display would have both colors (shown as arrows projecting from the screen). The eye covered by the first lens 30 would view the picture projected or displayed by the video source 24 projecting or displaying the opposite color. The eye covered by the second lens 32 would view the picture projected or displayed by the video source 26 projecting or displaying the opposite color. ChromaDepth® produces glasses (using micro prisms with similar two-color technology) that work on the same basic principle.
FIG. 2 shows an exemplary polarization differentiation system 40 for projecting or displaying three-dimensional video 42 that uses polarization for differentiation. The video 42 in this figure is shown with an exaggerated double image. This system takes advantage of the fact that polarized light will pass through polarized glass only if they are both polarized in the same direction. Accordingly, the first video source 44 projects through a first polarized filter (e.g. horizontal) and a second video source 46 projects through a second polarized filter (e.g. vertical). Viewers wear special glasses 48 with corresponding polarized lenses 50, 52. For example, the first lens 50 would have the same polarization as one of the polarized filters (e.g. shown as vertical dashed lines) and the second lens 52 would have the same polarization as the other polarized filter (e.g. shown as horizontal dashed lines). In this example, the eye covered by the first lens 50 would view the picture projected or displayed by the video source 44 projecting the horizontally polarized picture and the eye covered by the second lens 52 would view the picture projected or displayed by the video source 46 projecting the vertically polarized picture.
Another technology that is used for showing three-dimensional movies uses LCD shutter glasses. LCD shutter glasses have lenses that use liquid crystals and a polarizing filter that is transparent until a voltage is applied, at which time they become dark. An IR emitter sends an IR signal to trigger the voltage so that the lenses switch between transparent and dark in an alternating fashion, first one eye and then the other. This transparent/dark alternating is synchronized with the refresh rate of a specialized display screen that alternates between the display of a first perspective for a first eye and a second display for the second eye using a technique called alternate-frame sequencing. Used together, the LCD shutter glasses and the specialized display screen create the illusion of a three-dimensional picture (or at least three-dimensional elements of a picture).
Three-dimensional movies have been around for a long time. But after their heyday in the 1950s, three-dimensional movies as a medium fell into decline and movie producers turned their attention to other technologies. But new technologies (including polarization differentiation systems) have made this medium more attractive and new movies are being made and released as three-dimensional movies. A primary reason for this is that there has been a significant improvement in the quality of three-dimensional movies. Another reason that three-dimensional movies are becoming popular is that the movie viewing public appears willing to pay a premium for this special effect.
Even if movie producers are willing to invest in new technologies for producing new three-dimensional movies, it is significantly more expensive to film a movie using three-dimensional technology as compared to using two-dimensional technology. In addition, there are millions of two-dimensional movies that have already been produced. So there is a need to find a system or method for converting two-dimensional movies to three-dimensional movies.
Seeing this need, inventors have been trying to create methods and 30 systems for converting two-dimensional movies to three-dimensional movies. For example, there are many patents directed to methods and systems for converting two-dimensional movies for three-dimensional viewing. Many of these patents describe some type of analysis to “identify,” “cut out,” and/or “shift” one or more elements or objects in a scene and then layer the elements or objects to create the illusion of depth. Patents that fall into this category include, but are not limited to U.S. Pat. No. 6,477,267 to Richards and U.S. Pat. No. 7,321,374 to Naske. These prior art methods to convert two-dimensional movies for three-dimensional (stereoscopic) viewing, however, do not work at all (i.e. in theory they might work, but in practice they cannot work because currently available computer technology is not powerful enough to implement these computationally intensive methods), are resource intensive, and/or do not produce acceptable results (e.g. a cardboard cut-out effect). For example, some prior art methods are so computationally intensive that current processors are not powerful enough to handle the computations for more than a few elements or objects in each given scene.